Digital Filtering Algorithms for Decorrelation within Large Least Squares Problems

The GOCE (Gravity Field and steady-state Ocean Circulation Explorer) mission is dedicated to the determination of the Earth's gravity field. During the mission period of at least one year the GOCE satellite will collect approximately 100 million highly correlated observations. The gravity field will be described in terms of approximately 70,000 spherical harmonic coefficients. This leads to a least squares adjustment, in which the design matrix occupies 51 terabytes while the covariance matrix of the observations requires 72,760 terabytes of memory. The very large design matrix is typically computed in parallel using supercomputers like the JUMP (Juelich Multi Processor) supercomputer in Jülich, Germany. However, such a brute force approach does not work for the covariance matrix. Here, we have to exploit certain features of the observations, e.g. that the observations can be interpreted as a stationary time series. This allows for a very sparse representation of the covariance matrix by digital filters. This thesis is concerned with the use of digital filters for decorrelation within large least squares problems. First, it is analyzed, which conditions the observations must meet, such that digital filters can be used to represent their covariance matrix. After that, different filter implementations are introduced and compared with each other, especially with respect to the calculation time of filtering. This is of special concern, as for many applications the very large design matrix has to be filtered at least once. One special problem arising by the use of digital filters is the so-called warm-up effect. For the first time, methods are developed in this thesis for determining the length of the effect and for avoiding this effect. Next, a new algorithm is developed to deal with the problem of short data gaps within the observation time series. Finally, it is investigated which filter methods are best adopted for the application scenario GOCE, and several numerical simulations are performed.Digitale Filteralgorithmen zur Dekorrelation in großen kleinste-Quadrate Problemen Die GOCE (Gravity Field and steady-state Ocean Circulation Explorer) Mission ist der Bestimmung des Erdschwerefeldes gewidmet. Während der Missionsdauer von mindestens einem Jahr wird der GOCE Satellit circa 100 Millionen hoch korrelierte Beobachtungen sammeln. Das Erdschwerefeld wird durch circa 70.000 sphärisch harmonische Koeffizienten beschrieben. Dies führt zu einem kleinste-Quadrate Ausgleich, wobei die Designmatrix 51 Terabytes benötigt während die Kovarianzmatrix der Beobachtungen 72.760 Terabytes erfordert. Die sehr große Designmatrix wird typischerweise parallel berechnet, wobei Supercomputer wie JUMP (Juelich Multi Processor) in Jülich (Deutschland) zum Einsatz kommen. Ein solcher Ansatz, bei dem das Problem durch geballte Rechenleistung gelöst wird, funktioniert bei der Kovarianzmatrix der Beobachtungen nicht mehr. Hier müssen bestimmte Eigenschaften der Beobachtungen ausgenutzt werden, z.B. dass die Beobachtungen als stationäre Zeitreihe aufgefasst werden können. Dies ermöglicht es die Kovarianzmatrix durch digitale Filter zu repräsentieren. Diese Arbeit beschäftigt sich mit der Nutzung von digitalen Filtern zur Dekorrelation in großen kleinste-Quadrate Problemen. Zuerst wird analysiert, welche Bedingungen die Beobachtungen erfüllen müssen, damit digitale Filter zur Repräsentation ihrer Kovarianzmatrix benutzt werden können. Danach werden verschiedene Filterimplementierungen vorgestellt und miteinander verglichen, wobei spezielles Augenmerk auf die Rechenzeit für das Filtern gelegt wird. Dies ist von besonderer Bedeutung, da in vielen Anwendungen die sehr große Designmatrix mindestens einmal gefiltert werden muss. Ein spezielles Problem, welches beim Benutzen der Filter entsteht, ist der sogenannte Warmlaufzeiteffekt. Zum ersten Mal werden in dieser Arbeit Methoden entwickelt, um die Länge des Effekts zu bestimmen und um den Effekt zu vermeiden. Als Nächstes wird ein neuer Algorithmus zur Lösung des Problems von kurzen Datenlücken in der Beobachtungszeitreihe entwickelt. Schließlich wird untersucht, welche Filtermethoden man am besten für das Anwendungsszenario GOCE verwendet und es werden verschiedene numerische Simulationen durchgeführt

TOLEOS: Thermosphere Observations from Low-Earth Orbiting Satellites

The objective of the TOLEOS project is to process the CHAMP, GRACE, and GRACE-FO accelerometer measurements with improved processing standards to obtain thermosphere density and crosswind data products. These new data products will cover the entirety of the accelerometer missions and complement the existing ESA databases for Swarm and GOCE. The improvements in the processing focus on the radiation pressure modelling, which is expected to have a significant effect on the density and crosswind data, in particular at altitudes above 450 km during solar minimum conditions. Substantial validation activities are performed since the project’s start in June 2021 and will continue until the end of the project in July 2022

Preliminary Validation of Thermosphere Observations from the TOLEOS Project

OBSERVATIONS of upper atmospheric neutral mass density (NMD) and wind are critical to understand the coupling mechanisms between Earth’s ionosphere, thermosphere, and magnetosphere. The ongoing Swarm DISC (data, innovation, and science cluster) project TOLEOS (thermosphere observations from low-Earth orbiting satellites) aims to provide better calibrated NMD and crosswind data from CHAMP, GRACE, and GRACE-FO (follow-on) satellite missions. The project uses state-of-the-art models, calibration techniques, and processing standards to improve the accuracy of these data products and ensure inter-mission consistency. Here, we present preliminary results of the quality of the data in comparison to the high accuracy drag temperature model DTM2020, and physics-based TIE-GCM (thermosphere ionosphere electrodynamics general circulation model) and CTIPe (coupled thermosphere ionosphere plasmasphere electrodynamics) models

Lower-thermosphere–ionosphere (LTI) quantities: current status of measuring techniques and models

The lower-thermosphere-ionosphere (LTI) system consists of the upper atmosphere and the lower part of the ionosphere and as such comprises a complex system coupled to both the atmosphere below and space above. The atmospheric part of the LTI is dominated by laws of continuum fluid dynamics and chemistry, while the ionosphere is a plasma system controlled by electromagnetic forces driven by the magnetosphere, the solar wind, as well as the wind dynamo. The LTI is hence a domain controlled by many different physical processes. However, systematic in situ measurements within this region are severely lacking, although the LTI is located only 80 to 200 km above the surface of our planet. This paper reviews the current state of the art in measuring the LTI, either in situ or by several different remote-sensing methods. We begin by outlining the open questions within the LTI requiring high-quality in situ measurements, before reviewing directly observable parameters and their most important derivatives. The motivation for this review has arisen from the recent retention of the Daedalus mission as one among three competing mission candidates within the European Space Agency (ESA) Earth Explorer 10 Programme. However, this paper intends to cover the LTI parameters such that it can be used as a background scientific reference for any mission targeting in situ observations of the LTI.Peer reviewe

Digital filtering algorithms for decorrelation within large least squares problems

The GOCE (Gravity Field and steady-state Ocean Circulation Explorer) mission is dedicated to the determination of the Earth's gravity field. During the mission period of at least one year the GOCE satellite will collect approximately 100 million highly correlated observations. The gravity field will be described in terms of approximately 70,000 spherical harmonic coefficients. This leads to a least squares adjustment, in which the design matrix occupies 51 terabytes while the covariance matrix of the observations requires 72,760 terabytes of memory. The very large design matrix is typically computed in parallel using supercomputers like the JUMP (Juelich Multi Processor) supercomputer in Jülich, Germany. However, such a brute force approach does not work for the covariance matrix. Here, we have to exploit certain features of the observations, e.g. that the observations can be interpreted as a stationary time series. This allows for a very sparse representation of the covariance matrix by digital filters. This thesis is concerned with the use of digital filters for decorrelation within large least squares problems. First, it is analyzed, which conditions the observations must meet, such that digital filters can be used to represent their covariance matrix. After that, different filter implementations are introduced and compared with each other, especially with respect to the calculation time of filtering. This is of special concern, as for many applications the very large design matrix has to be filtered at least once. One special problem arising by the use of digital filters is the so-called warm-up effect. For the first time, methods are developed in this thesis for determining the length of the effect and for avoiding this effect. Next, a new algorithm is developed to deal with the problem of short data gaps within the observation time series. Finally, it is investigated which filter methods are best adopted for the application scenario GOCE, and several numerical simulations are performedDigitale Filteralgorithmen zur Dekorrelation in großen kleinste-Quadrate Problemen Die GOCE (Gravity Field and steady-state Ocean Circulation Explorer) Mission ist der Bestimmung des Erdschwerefeldes gewidmet. Während der Missionsdauer von mindestens einem Jahr wird der GOCE Satellit circa 100 Millionen hoch korrelierte Beobachtungen sammeln. Das Erdschwerefeld wird durch circa 70.000 sphärisch harmonische Koeffizienten beschrieben. Dies führt zu einem kleinste-Quadrate Ausgleich, wobei die Designmatrix 51 Terabytes benötigt während die Kovarianzmatrix der Beobachtungen 72.760 Terabytes erfordert. Die sehr große Designmatrix wird typischerweise parallel berechnet, wobei Supercomputer wie JUMP (Juelich Multi Processor) in Jülich (Deutschland) zum Einsatz kommen. Ein solcher Ansatz, bei dem das Problem durch geballte Rechenleistung gelöst wird, funktioniert bei der Kovarianzmatrix der Beobachtungen nicht mehr. Hier müssen bestimmte Eigenschaften der Beobachtungen ausgenutzt werden, z.B. dass die Beobachtungen als stationäre Zeitreihe aufgefasst werden können. Dies ermöglicht es die Kovarianzmatrix durch digitale Filter zu repräsentieren. Diese Arbeit beschäftigt sich mit der Nutzung von digitalen Filtern zur Dekorrelation in großen kleinste-Quadrate Problemen. Zuerst wird analysiert, welche Bedingungen die Beobachtungen erfüllen müssen, damit digitale Filter zur Repräsentation ihrer Kovarianzmatrix benutzt werden können. Danach werden verschiedene Filterimplementierungen vorgestellt und miteinander verglichen, wobei spezielles Augenmerk auf die Rechenzeit für das Filtern gelegt wird. Dies ist von besonderer Bedeutung, da in vielen Anwendungen die sehr große Designmatrix mindestens einmal gefiltert werden muss. Ein spezielles Problem, welches beim Benutzen der Filter entsteht, ist der sogenannte Warmlaufzeiteffekt. Zum ersten Mal werden in dieser Arbeit Methoden entwickelt, um die Länge des Effekts zu bestimmen und um den Effekt zu vermeiden. Als Nächstes wird ein neuer Algorithmus zur Lösung des Problems von kurzen Datenlücken in der Beobachtungszeitreihe entwickelt. Schließlich wird untersucht, welche Filtermethoden man am besten für das Anwendungsszenario GOCE verwendet und es werden verschiedene numerische Simulationen durchgeführt

Swarm in-situ neutral horizontal winds based on Swarm plasma and neutral density measurements

In this presentation, we provide preliminary results of a new thermosphere neutral wind data product along the Swarm satellites. Thermosphere wind measurements are sparse in general, and in-situ measurements are nonexistent along Swarm. The neutral winds are key to understand the thermosphere-ionosphere dynamics. Thermospheric neutral winds mostly below 250 km in altitude have been observed by Fabry–Pérot interferometers during nighttime, and derived from incoherent scatter radar measurements for decades, but only at a few stations. In-situ measurements of neutral winds also exist covering various altitudes mostly below 400 km, but only for a limited period (e.g. GOCE and CHAMP satellites). Characterisation of the winds, for example, in terms of latitudinal, diurnal, seasonal, and solar cycle variations is a difficult task with these limited data. The literature, in general, also suggest disagreements between these data products not only by the technique used to derive the winds but also by the station. Unsurprisingly, the complex interconnectedness of neutral winds in upper atmospheric dynamics is not fully understood. Here, we use existing Swarm data products (neutral mass density, plasma density, and ion drift velocities) to derive the horizontal components of the in-situ neutral wind. We evaluate the new prototype data product against both empirical and physical models. We discuss the opportunity for a uniquely new data product with over 8 years (and continuing) of Swarm data to characterise neutral winds at all local times at its different altitudes, which start above 500 km